Relevance feedback on a segment of a data object

ABSTRACT

The invention relates to a method, a system ( 101 ) and a computer program product to identify a particular data object of a data type in a database ( 104 ) that comprises data objects of the data type. The system ( 101 ) comprises a query composition unit ( 102 ) to compose a search query to identify a candidate data object being a candidate for the particular data object. A search unit ( 103 ) identifies the candidate data object in the database ( 104 ) based on the search query. A presentation unit ( 105 ) presents the candidate data object to the user. A feedback receiving unit ( 106 ) receives user feedback on the relevance or irrelevance of a segment of the candidate data object. The search unit ( 103 ) further identifies an improved candidate data object in response to the received user feedback. The improved candidate data object is an improved candidate for the particular data object.

FIELD OF THE INVENTION

The invention relates to the field information retrieval and morespecifically to the field of systems, methods and computer programproducts for identifying a particular data object of data type in adatabase comprising data objects of the data type.

BACKGROUND OF THE INVENTION

Information retrieval systems search in a database to retrieveinformation the user is looking for. In information retrieval systemsthe biggest challenge is to understand the user's needs. A system basedon keyword search often has a limitation in the query language andindexation of the database. For the user it is difficult to expressaccurately in keywords what he is looking for. Information retrieval inan image based database is even more difficult, because it is difficultfor systems to interpret an image in the same manner as a human being. Asuccessful paradigm to solve this problem has been “relevance feedback”.Such a system retrieves candidate data objects that match the searchquery, the user reviews the retrieved candidate data objects andprovides user feedback on the relevance or irrelevance of the candidatedata objects. The system learns from the user feedback to improve itssearch performance. If a candidate data object is similar to aparticular data object for which the user is searching, the user selectsthe candidate data object as being relevant for the search. If acandidate data object is not similar to the user's intensions or even acompletely wrong search result, the user selects the data object asbeing irrelevant for the search. Thus, the system learns from thefeedback that it has to search for data objects that are similar to therelevant candidate data objects and that are not similar to theirrelevant candidate data objects.

A system for image retrieval is described in an article by Qi Tian, etal, entitled “Combine user defined region of interest and spatial layoutfor image retrieval”, in Proc. of IEEE 2000 International Conference onImage Processing (ICIP'2000), pp. 746-749, Vol. 3, Vancouver, BC,Canada, Sep. 10-13, 2000. The system disclosed in the reference documenttries to find image objects in a database of image objects based on aquery image. The system has to find images that are similar to the queryimage. The images in the database are subdivided in a grid ofpre-defined regions. For every region a feature vector is present in thedatabase. The system compares the query image with the images in thedatabase to find images that have feature vectors for regions similar tothe feature vectors of the regions of the query image. The images in thedata with most similar feature vectors are presented to the user asreturn images. Return images are candidates for the image, or images,the user is looking for.

Subsequently the user has to provide relevance feedback on a number ofreturn images such that the system learns about the intentions of theuser. This feedback triggers an additional search wherein the systemtries to find images that better match the intentions of the user. Aftera number of iterations a set of images is found by the system bestmatching the intentions of the user. In order to allow the system tooptimally learn the intentions of the user, the system requires feedbackon many return images and thus many iterations are required.

In addition, the system offers the user the possibility to definetogether with the query image a Region Of Interest (ROI). The userindicates to the system which part of the image has his interest andindicates as such that the system has to find similarities between thequery image and the image in the database inside the ROI only.Information in the query image outside the ROI is not of interest to theuser and needs to be ignored by the system. The system uses the ROI todetermine weight values for the regions of the grid of regions. Regionsthat fall completely outside the ROI get a weight value of 0. Regionsthat partly overlap with the ROI get a lower weight value than theregions that completely overlap with the ROI. Regions with a low weightvalue will only marginally be taken into account in the search, andregions with a high weight value will influence the search results most.Each iteration the system uses the same set of weight values for thesearches. Although the focus of the search is marginally better, theuser has still to provide feedback on many images.

Reviewing many return images to provide feedback regarding the returnimage and going through many iterations is especially a disadvantage inthe medical domain where the users of the systems are medical expertswho have limited time and resources. Furthermore, in the case that theimages need to be transmitted to a handheld device for receivingfeedback, for example, to a handheld device that the medical expert useswhen visiting patients, the number of wireless transmissions must beminimized because of limited available bandwidth for the handhelddevice.

SUMMARY OF THE INVENTION

It is an object of the invention to reduce the number of reviews ofreturn images that are presented to the user.

The invention is defined by independent claims. Advantageous embodimentsare defined in the dependent claims.

A first aspect of the invention provides a system as claimed in claim 1.A second aspect of the invention provides a method as claimed in claim14. A third aspect of the invention provides a computer program productas claimed in claim 15.

The system in according to the first aspect of the invention providesthe user with a system that comprises a query composition unit. Thequery composition unit composes a search query to identify a candidatedata object such that the candidate data object is a candidate for theparticular data object. The search query is used to inform the searchunit about the intentions of the user. The user expects from the systemto identify a particular data object in the database. The search querymay be based on an example data object and the system has to identify aparticular object that is similar to the example data object. The searchquery may also be based on keywords in which the intentions of the userare expressed in words. Alternatively, a combination of an example dataobject and keywords are used.

The system further comprises a search unit that identifies the candidatedata object in the database based on the search query. The search unitmay for example identify the candidate data object in the databasebecause it has most similar keywords to the keywords of the searchquery. Or the search unit may, for example, identify the candidate dataobject that is most similar to the example data object of the searchquery. If the data objects are images, the similarity is for examplerelated to average intensity, contrast, intensity distribution,distributions of the color tones of the pixels of the primary colors inthe RGB color model or for example the distributions of the values ofthe pixels of the dimensions in other color models.

The system presents the data object to the user with a presentationunit. The presentation unit may comprise a display to display forexample an image or a video fragment and the presentation unit may alsocomprise an amplifier with loudspeakers to play an audio fragment to theuser or to play the audio of a video fragment.

The feedback receiving unit of the system receives user feedback on therelevance or irrelevance of a segment of the candidate data object. Theuser provides feedback to the feedback receiving unit. The feedback maybe that the segment of the candidate data object is irrelevant, whichmeans that the user indicates to the system that the segment containsinformation that does not match his expectations concerning theparticular data object. The feedback may be that the segment of thecandidate data object is relevant, which means that the user indicatesto the system that the segment contains information that does match hisexpectations.

Subsequently, the search unit of the system identifies an improvedcandidate data object in response to the received user feedback. Theimproved candidate data object is an improved candidate for theparticular data object. The system learns from the received feedback andtherefore the search unit is able to find a better match in the databasein relation to the intentions of the user.

The user of the system is able to provide feedback on segments of thecandidate data object. A segment is a part of the data object. Thesegment is for example a spatial partition of a two or three dimensionaldata object or a temporal partition of the data object that comprises atime line. The spatial decomposition is for example a part of forexample a medical image, or a sub volume of volumetric medical data. Acandidate data object may be partitioned in time if the data object isfor example video or audio data. The user is able to define whether thesegment of the candidate data object is according to his opinionrelevant or irrelevant. The advantage of relevance feedback on thesegment is that the user feedback on a segment is much more accurate andexpresses the intentions of the user much better than feedback on thewhole data object.

If a candidate data object is presented to the user and the user is notcompletely satisfied with the presented candidate data object, the userdoes not accept the candidate data object as the data object he issearching for, which is the particular data object. The rejection ismost probably based on the fact that not the whole data object, but onlya segment of the candidate data object is seen as relevant orirrelevant. Providing relevance or irrelevance feedback on the wholedata object ignores the fact that only a part of the candidate dataobject is the reason for rejecting or accepting the candidate dataobject as the particular data object. Therefore, feedback on the wholecandidate data object is less accurate. Thus, systems that requirefeedback on the whole data object converge less quickly to a candidatedata object that is accepted by the user as the data object he issearching for. Consequently, the system in accordance with the firstaspect of the invention requires fewer reviews of data objects forproviding feedback.

In particular in the medical domain providing feedback on a segment ofthe data object is very important because different parts of medicalimages provide cues for different pathologies. Thus, by providingfeedback on specific segments, the system focuses more on specificpathologies.

It should be noted that providing user feedback on the relevance orirrelevance of a segment of the candidate object is conceptuallydifferent from the Region Of Interest (ROI) of the cited state of theart. With the ROI the user tells the system immediately in the initialsearch query that only a region of the query image must be matched withthe same region of the images in the database. As such the user informsthe system that it has to temporarily throw away the information in thearea outside the ROI of all pictures in the database because the areaoutside the ROI should be ignored. Thus, the ROI of the query image isthe only relevant part of the query image. The system of the cited artuses the ROI to update the weight values of the regions of the grid ofregions. The updated weight values of the regions are used by the systemfor all searches that are executed—also for the searches after receivingrelevance feedback. In the system of the cited art the user can onlydecide about the relevance or irrelevance of every return image as awhole.

An example of a system in accordance with the first aspect of theinvention is a system with a medical database comprising MRI brain scansof patients suffering different forms of dementia and MRI brain scans ofa control group of healthy patients. A doctor who obtains an MRI scan ofthe brain of one of his patients suffering dementia wants to identify inthe database a similar MRI brain scan. The initial search query iscomposed on basis of the obtained MRI brain scan of the patient. Thesystem finds in the database one or more candidate MRI brain scans thatare similar to the MRI brain scan in the search query. These MRI brainscans are presented to the doctor. The doctor reviews the candidate MRIbrain scans and provides the system with his feedback on at least onesegment of at least one of the candidate MRI brain scans. The doctorindicates whether the segment of the candidate MRI brain scan isrelevant or irrelevant for the result he is looking for. For example,the doctor may decide that in an area of the presented candidate MRIbrain scan a structure is present which is not at all similar to theinitial provided MRI brain scan of the patient. Therefore the doctorprovides for this area the user feedback “irrelevant”. It may also bethe case that the presented candidate MRI brain scan looks healthy in aspecific part of the MRI brain scan, while the MRI brain scan of thepatient does not look healthy in the specific part of the MRI brainscan. Thus user feedback “irrelevant” is provided for the specific partof the candidate MRI brain scan. And the doctor may decide that in oneof the presented candidate MRI brain scans an area is very similar tothe MRI brain scan of his patient and that the area is related to one ofthe possible forms of dementia (for example Alzheimer's disease). Thus,the user feedback “relevant” is provided for the area that is verysimilar. Subsequently, the system updates the search in response to thereceived feedback on one or more segments and presents another list ofone or more candidate MRI brain scans. Note that some of the originallypresented MRI brain scans may still be in the result list after updatingthe search results just because they are simply very good matches.

The system in accordance with the first aspect of the invention, themethod in accordance with the second aspect of the invention and thecomputer program product provide the user with the same benefits inrelation to the reduction of the number of reviews of candidate dataobjects.

In an embodiment the user defines the segment of the candidate dataobject. The user indicates to the feedback receiving unit which part ofthe candidate data object is the segment. It is an advantage for theuser to select the segment because in that way the user can be mostaccurate in his feedback. The user may, to define the segment, forexample, select an area in the form of a polygon or a volume in the formof polyhedron. By receiving more accurate user feedback the systemidentifies an improved candidate object that better fulfils theexpectations of the user and as such there is a faster convergence. Itshould be noted that the segment may also be another geometric shape,like a circle or a sphere.

In another embodiment the system further comprises a segment proposalunit. The segment proposal unit proposes a set of candidate segments.The candidate segments are parts of the candidate data object and are acandidate for the segment on which the user provides relevance feedback.The candidate segments are presented to the user by the presentationunit together with the candidate data object. Subsequently the feedbackreceiving unit receives from the user an indication which of thecandidate segments is the segment on which the user wants to providerelevance or irrelevance feedback. Selecting one of the proposedcandidate segments may be done quickly and as such it takes not muchtime for the user to provide the feedback. This especially an advantageif the user does not have much time to provide feedback to the system.

In an embodiment the data object is a medical data object, like an imageof the body obtained with X-ray photography, Computer Tomography (CT) orMagnetic Resonance Imaging (MRI), a three dimensional representation ofa part of the body of a patient obtained by combining several CT or MRIimages, or a film in which the circulation of the blood through thevessels and the heart of a patient is visualized.

The segment proposal unit is constructed to recognize anatomicallydefined parts of the body in the media data object. The anatomicallydefined parts of the body are based on, for example, medical atlas baseddefinitions. The anatomically defined parts are proposed as candidatesegments. In the medical domain the medical expert is often searchingfor specific characteristics of anatomically defined parts of the body,because specific anatomically defined parts of the body provide cues fordifferent pathologies. It is an advantage if the system presentspossible segments in combination with the candidate data object thatrelate to the anatomical parts of the body, such that the medical expertcan easily select the segment. This saves time for the medical expert.

In another embodiment the segment proposal unit proposes the set ofcandidate segments on basis of a partitioning of the data objectaccording to a grid. The proposed candidate segments may be a set ofpartially overlapping segments, or the candidate segments may bedisjoint. The whole set of candidate segments may cover the wholecandidate data object. It is an advantage to partition the candidatedata objects according to a grid, because it is a very fast andeffective way of partitioning the candidate object. It does not costmany system resources to perform the partitioning. Another advantage maybe that the data objects in the database have been partitioned insegments according to the same grid as well and that for every segmentsome additional information is stored. In the case that the databasestores additional information related to segments according to the samegrid, the search unit is able to identify much faster an improvedcandidate data object.

In an embodiment the feedback receiving unit receives user feedback onthe relevance or irrelevance of at least two segments of the candidatedata object. The more feedback is given related to different segments ofthe candidate data object, the more accurate the system may identify animproved candidate object, because the system learns more from theuser's intentions. Furthermore, especially in the medical domain themedical expert has to review the whole data object for providing userfeedback. While reviewing the whole data object the medical expertidentifies in general several segments that are relevant and severalsegments that are irrelevant. To use the reviewing time efficiently itis an advantage to provide all observations immediately to the system.As a consequence of more user feedback per candidate data object thesystem converges faster to a candidate data object that is accepted bythe user as the data object for which the initial search query wascomposed. Thus, the system requires fewer reviews of data objects andfewer iterations.

Note that the at least two segments of the candidate data object may(partially) overlap or may be disjoint. The at least two segments mayalso completely overlap, which is an advantage in the case ofhierarchical relations between some of the segments. An example of userfeedback in the case of hierarchy between the segments is: In the caseof a medical data object representing the human brain, used feedback toa segment related to the Basal Ganglia is received by the feedbackreceiving unit. The Basal Ganglia is composed of segments such as theGlobus Pallidus and the Putamen. And in addition to user feedback on thesegment related to the Basal Ganglia the user may also provide separatefeedback to a segment related to the Globus Pallidus and to a segmentrelated to the Putamen.

In a further embodiment the feedback receiving unit receives a rankingof the at least two segments of the candidate data object from the user.The ranking expresses which segment is more important for the user thanwhich other segment. The search unit uses this ranking internally todecide which of the data objects is the best improved candidate object.An expert who reviews the candidate object realizes immediately whichsegment is most important for identifying an improved data object andwhich segments are less important, but still important enough to providefeedback for. By providing user feedback in the form of the ranking ofsegments the system is better able to identify the improved data objectand as such the improved data object better matches the intentions ofthe user.

In an embodiment the feedback receiving unit receives user feedback onthe relevance or the irrelevance of a feature of the content of thesegment of the candidate data object. While reviewing the candidate dataobject the user has particular reasons why a segment is relevant orirrelevant. It is an advantage to express these reasons in the form of afeature that is related to the content of the segment of the candidatedata object. The user feedback in the form of feedback on the relevanceor irrelevance of the feature of the content of the segment provides thesystem much more accurate information about the intensions of the userand as such about the improved data object that must be identified. Forexample, in the medical domain the medical expert discovers during areview that segments of a candidate medical image is subject tohyperintensity or hypointensity, which means lighter and darker thanexpected, respectively. It may be relevant or it may be irrelevant forthe search that the content of a segment of the data object is subjectto hyperintensity or hypointensity. And in addition the search unit mayperform a faster identification of the improved data object if thedatabase contains also information about areas of the image that aresubject to hyperintensity or hypointensity. It should be noted that theuser may provide relevance feedback on more than one feature of thecontent of the segment. For example, the content of the segment may besubject to hyperintensity and the shape of an anatomical part of thebody in the segment is wrong.

In another embodiment the identification of the improved candidateobject is performed by updating the search query in response to thereceived user feedback. The updated search query is used to identify theimproved candidate object. There are known algorithms, like Rocchio'salgorithm for relevance feedback, to update the search query based onreceived relevance feedback. These algorithms are mainly based on thefact that the initial search query is described as a feature vector, forexample a feature vector that is based on the content of an exampleimage that the user provided to the query composing unit. The searchunit uses the feature vector to find data objects with similar featurevectors. The updating of the search query is done by combining theinitial feature vector with arithmetic operation with the featurevectors of the candidate objects on which feedback was provided. Insteadof a feature vector a feature matrix may be used, wherein, for example,the different columns are feature vectors related to different parts ofthe data object. Similarly to Rocchio's algorithm, the updated featurematrix is based on the initial feature matrix and with some arithmeticoperations some of the columns may be updated in response to thereceived user feedback on the segment.

In an embodiment the identification of the improved candidate dataobject is performed by identifying a first list of candidate dataobjects and identifying a second list of candidate data objects. Thefirst list of candidate data objects is identified on basis of theinitial search query. The second list of candidate data objects isidentified on basis of the received user feedback on the relevance orirrelevance of the segment. Subsequently, the improved candidate dataobject is selected from the first list or the second list.

By performing two identification actions two relative simple searcheshave to be executed. The first list of candidate objects may still bepresent in the search unit as a result of the initial identification ofthe candidate object. If the provided relevance feedback was positive(“relevant”), the second list of candidate data object has to containdata objects that have content in a part of the data object that issimilar to the content of the segment of the candidate object.Similarly, if the provided feedback was negative (“irrelevant”), thesecond list of candidate data object has to contain data objects thathave content in a part of the data object that is dissimilar to thecontent of the segment of the candidate object. Subsequently, theselection of the improved candidate object may be done on basis of afairly simple criterion, like “which data object appears in both lists”.Selecting the improved candidate object from the first or second listmay also be done by assigning to the individual data objects in both ofthe lists a score, merging the lists, and selecting the data object withthe highest score. The score has to express the similarity between thedata object and the initial search query, or the similarity between thepart of the data object and the segment of the candidate object in caseof positive feedback for the segment, or the dissimilarity between thepart of the data object and the segment of the candidate object in caseof negative feedback for the segment.

In the embodiments different variants of user feedback are discussed,like feedback on the relevance or irrelevance of a segment, therelevance or irrelevance of a feature of the contents of a segment and aranking of segments. It should be noted that different combinations ofthe variants of user feedback may be used, like providing user feedbackon the relevance or irrelevance of a feature of the contents of asegment, without providing user feedback on the relevance or irrelevanceof the segment.

In the embodiments different variants of segments are discussed, likeuser defined segments and user selected segments based on proposedanatomically defined segments or proposed segments based on a grid. Itshould be noted that the invention is not limited to the separate use ofthe variants of segments, but that different combinations of thevariants of segments may be used.

It will be appreciated by those skilled in the art that two or more ofthe above-mentioned embodiments, implementations, and/or aspects of theinvention may be combined in any way deemed useful.

Modifications and variations of the method, and/or of the computerprogram product, which correspond to the described modifications andvariations of the system, can be carried out by a person skilled in theart on the basis of the present description.

A person skilled in the art will appreciate that the method may beapplied to multidimensional image data, for example, to 2-dimensional(2-D), 3-dimensional (3-D) or 4-dimensional (4-D) images, acquired byvarious acquisition modalities such as, but not limited to, standardX-ray Imaging, Computed Tomography (CT), Magnetic Resonance Imaging(MRI), Ultrasound (US), Positron Emission Tomography (PET), SinglePhoton Emission Computed Tomography (SPECT), and Nuclear Medicine (NM).

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings:

FIG. 1 shows a schematic diagram of a system for identifying aparticular data object in a database,

FIG. 2 shows a schematic view of the display of the presentation unit ofthe system,

FIG. 3 shows a schematic view of a slice of a MRI brain scan withselected segments,

FIG. 4 shows a schematic view of a slice of a MRI brain scan withproposed candidate segments,

FIG. 5 shows a schematic view of a CT scan of the neck with proposedcandidate segments according to a grid,

FIG. 6 shows a schematic view of the presentation of a data type of avideo, and

FIG. 7 shows a flow diagram of a method for identifying a particulardata object in a database.

It should be noted that items which have the same reference numbers indifferent figures, have the same structural features and the samefunctions, or are the same signals. Where the function and/or structureof such an item has been explained, there is no necessity for repeatedexplanation thereof in the detailed description.

DETAILED DESCRIPTION

FIG. 1 shows a schematic diagram of a system 101 to identify aparticular image in a database 104. Although the embodiment is describedfor a database comprising images, the system may be used to identify aparticular data object of another data type. For example, threedimensional representations of objects or environments, video fragmentsor audio fragments.

The system comprises a query composition unit 102 to compose a searchquery. The search query expresses the intensions of the user in relationto the particular image; it expresses what kind of image the user islooking for. The search query is used by the system 101 to identify acandidate image that is a candidate for the particular image. Based onuser input the search query is composed. The user provides an exampleimage that is used to compose the search query. In another embodiment itmay be possible that the user provides keywords that describe thecontent of the image that he wants the system to identify.

The query composition unit 102 composes, based on the example image, asearch query in the form of a feature matrix. The feature matrixdescribes several characteristics of the example image for severalregions of the image. The columns of the feature matrix are featurevectors that describe characteristics of a part of the example image.The feature vector comprises for example the distribution of thedifferent color tones in the R, G and B channel of the part of theexample image.

The search unit 103 of the system 101 identifies, based on the searchquery, the candidate image in the database 104. This is done bycomparing the feature matrices of the images in the database 104 withthe feature matrix of the search query. Note that the database 104 maycomprise the feature matrices of the images in the database 104 in casethat the feature matrices were created at the moment that the database104 was created. The search unit 103 may also extract the featurematrices from the images in the database 104 while the search unit 103is identifying the candidate image. The identified candidate image isthe image in the database 104 with the feature matrix that is mostsimilar to the feature matrix of the search query.

The system 101 has a presentation unit 105 that presents the candidateimage to the user. The presentation unit 105 comprises a display thatdisplays the candidate image. The presentation unit may comprise otherdata presentation equipment, like loudspeakers to play an audiofragment.

The user feedback receiving unit 106 of the system receives userfeedback for a segment of the candidate image. The user indicateswhether the segment of the candidate image is relevant or irrelevant. Ifthe segment is relevant, the user is looking for a particular image thathas similar characteristics to the characteristics of the segment of thecandidate image. If the segment is irrelevant, the user is looking for aparticular image that has in the segment dissimilar characteristics tothe characteristics of the segment of the candidate image. The feedbackreceiving unit 106 may comprise an input receiving device that is usedby the user to define the segment of the candidate image and to indicatethe relevance or irrelevance of the segment. If the display of thepresentation unit 105 is for example a touch screen and the feedbackreceiving unit 106 is coupled to the touch detecting device of the touchscreen. This allows the user to select on the touch screen a segment ofthe presented candidate image. Another input receiving device is forexample a mouse.

Subsequently, the received user feedback at the feedback receiving unit106 is used by the search unit 103 to identify an improved candidateimage in the database 104 that better fulfils the intentions of theuser. The search unit 103 may for example update the feature matrix ofthe search query by changing some of the columns of the feature matrix.These columns are changed on basis of the feature vector of the segmentof the candidate image. Which columns are updated depends on the overlapof the segment with the parts of the images that are represented withseparated feature vectors. How much the columns are updated depends onthe amount of overlap between the segment and the parts of the image.

The updated feature matrix is used by the search unit 103 to identifythe improved candidate image in the database 104 that has a featurematrix that is most similar to the updated feature matrix. Note that theimproved candidate image may be another image than the initiallyidentified candidate image, if the another image has a feature matrixthat is most similar to the updated feature matrix. Note that theimproved candidate image may be the same image as the candidate image ifthe feature matrix of the previously found candidate image is still mostsimilar to the updated feature matrix.

In another embodiment the system comprises a segment proposal unit 107.The segment proposal unit 107 proposes for the candidate image a set ofcandidate segments. The set of candidate segments is presented by thepresentation unit 105 to the user. The display of the presentation unit105 may display areas of the different candidate segments in anothercolor, or display the content of the area blinking, or draw a line of aspecific color around the candidate segments. The feedback receivingunit 106 receives from the user a selection of candidate segments asbeing the segment on which the user provides feedback. The segmentproposal unit 107 may subdivide the candidate image in disjointcandidate segments on basis of a grid. In the case that the images aremedical images, the segment proposal unit may have knowledge about theshown parts of the body and may propose segments based on anatomicallydefined segment. In another embodiment the segment proposal unit 107 maycomprise an edge detector to detect edges in the image that areboundaries of parts of the body. The segment proposal unit 107 may usethe detected boundaries of parts of the body to propose segments.

FIG. 2 shows picture 201 displayed on a display of the presentation unit105 according to an embodiment. Example image 202 is a slice, in coronalview, of a MRI brain scan is shown. Furthermore, a first candidate image206 and a second candidate image 207 are shown.

The example image 202 was used by the query composition unit 102 tocompose the search query. The system 101 has to find other slices incoronal view of a MRI brain scans in the database 104 that are similarto the example image 202. A medical expert may realize that theventricles 203 and 204 are enlarged and that the size of cerebral cortex205 on the right upper side of the example image 202 is slightlydecreased compared with the left side of the brain.

The search unit 103 has identified two candidate images 206 and 207 thatare similar to the example image 202. The medical expert may realizeduring revision of the first example image 206 that the hippocampi 208and 209 are much smaller than usual, that the ventricle 209 has thenormal size, and that the cerebral cortex at the right upper side of theimage of the first candidate image 206 has been shrunk (and even morethan the shrinkage in the example image 202). The second candidate image207 shows a slice of a brain scan in which the ventricles 211 and 212are enlarged and in which the cerebral cortex 213 has the usual size.The user feedback receiving unit 106 receives feedback on segments ofthe candidate images. This is shown in FIG. 3 and FIG. 4.

The slices of the MRI brain scan are shown in coronal view. In anotherembodiment slices in an axial view and/or sagittal view are used by thesystem 101 to identify a candidate slice in axial and/or sagittal viewin the database 104. In another embodiment the shown MRI brain scan maybe a volumetric representation, which is a three dimensional view.

In FIG. 3 the first candidate image 206 is shown. The medical expert whois using the system 101 defines in the first candidate image 206 foursegments 301, 302, 303 and 304. The medical expert indicates to thefeedback receiving unit 106 that the segment 301 is irrelevant becausein his opinion the ventricle 209 is not enlarged in comparison to theventricle 204 of the example image 202. The medical expert indicatesthat segment 302 is relevant because in his opinion is the shrinkage ofthe cerebral cortex 210 is similar to the shrinkage of the cerebralcortex 205 of the example image 202. Furthermore the medical expertprovides irrelevance feedback for the segments 303 and 304, because inhis opinion he is not looking for a particular image in which thehippocampi are smaller than usual.

In an embodiment the medical expert is able to provide additionalfeedback to the feedback receiving unit 106 in the form of a ranking ofthe segments 301, 302, 303 and 304. If it is much more important for themedical expert that the cerebral cortex 210 is subject to shrinkage,than the fact that the hippocampi 208 are not subject to shrinkage, themedical expert may provide a ranking of (from most important to leastimportant): 302, 301, 303, 304.

Segments 301 to 304 are a spatial partitioning of the candidate image206. In another embodiment the data objects in the database are threedimensional representations of the brain. The presentation unit isadapted to present the three dimensional images of the brain and offersthe user a user interface in which the user is able to rotate thedisplayed image and to look inside the structure of the brain. Thefeedback receiving unit receives, in the case of a three dimensionalrepresentation, relevance feedback on a segment that is a volumetricpartitioning of the three dimensional representation of the brain.

FIG. 4 shows the second candidate image 207. In FIG. 4, according to afurther embodiment, the segment proposal unit 107 proposed candidatesegments 401 and 402. The candidate segments 401 and 402 are selected bythe segment proposal unit 107 on basis of anatomically defined parts ofthe brain, for example each of the two candidate segments 402 is relatedto a ventricle. The medical expert who reviews the image realizes thathe wants to give his feedback for the candidate segments 402 becausethey cover the part of the image in which the ventricles 211 and 212 areenlarged. Consequently, the medical expert provides the feedbackreceiving unit 106 with an indication that he wants to select candidatesegments 402 as the segments on which he provides relevance feedback. Inaddition the medical expert of the system 101 defines segment 403. Thefeedback receiving unit 106 further receives negative feedback(irrelevant) for segment 403 and positive (relevant) feedback forsegments 402.

In FIG. 5 shows a candidate image 501 that is presented on the displayof the presentation unit 105. The candidate image 501 is generated onbasis of several CT scans of the neck and head of a patient. Amultiplanar reconstruction technique is used to create a cross-cut ofthe neck and lower part of the head at the vertebral column seen fromthe flank (this is a sagittal view of the neck and the lower part of thehead).

The segment proposal unit 107 proposes candidate segments on basis of asubdivision of the image according to a grid. The candidate segments arepresented by the presentation unit 105 to the user. As can be seen inFIG. 5 the candidate image 501 is subdivided into rows 502 and columns503. The user that reviews the candidate image indicates to the feedbackreceiving unit 106 that he wants to provide feedback to the candidatesegments 504 to 506. Subsequently the user indicates that segment 504 isirrelevant, which is shown with the indication 507. The display of thepresentation unit 105 shows the “negative” indication 507. The userindicates that segments 505 and 506 are relevant, which is shown by the“positive” indications 508 and 509.

In a further embodiment the user provides relevance feedback on thefeatures of contents of the segments. If, for example, the candidateimage 501 has in segment 505 a higher average intensity than expected,which is called hyperintensity, the user provides the feedback receivingunit the feedback that the hyperintensity segment 505 is relevant orirrelevant. If the hyperintensity of a segment is relevant, it means forthe system that it has to search for improved candidate images with anequal average intensity in the part of the improved candidate image thatmatches with the average intensity of the segment on which feedback wasprovided. If the hyperintensity of a segment is irrelevant it means thatthe improved candidate image has to have a lower intensity in the partof the image than the intensity in the segment to which the relevancefeedback relates.

In another embodiment, discussed together with FIG. 6, the data objectsin the database 104 may be video fragments of pulsations of the humanheart. The medical expert is looking for a video that shows a specificbehavior of one of the valves 603 of the heart. Based on a search querythe search unit 103 found a candidate video fragment that is presentedto the medical expert. FIG. 6 shows in a display image 601 a userinterface of the system 101 that is presented on the display of thepresentation unit 105 at the moment that the presentation unit 105presents the candidate video fragment. To review the candidate videofragment the user interface provides the user with a video player. Inwindow 602 the candidate video fragment is presented. The user interfaceshows time line 604. Indicator 607 shows where the currently shown videoimage in window 602 is located on the time line. The feedback receivingunit 106 receives via the user interface relevance feedback on a segmentof the candidate video fragment. The segment is a part of the time lineof the candidate video fragment and as such the segment is a temporalpartition of the candidate video fragment. The user interface in FIG. 6shows that the medical expert defined a segment of the time line thatstarts at indication 605 and ends at indication 608. As can be seen bythe positive indicator 606 the medical expert provided positivefeedback.

In another embodiment the defined segment of FIG. 6 is combined with aspatial partition of the video fragment because the medical expert wantto have the search more focused on a specific part of the heart, forexample around valve 603. A spatial subdivision of a video fragmentmeans that the segment defines a part of each of the images in a set ofconsecutive video images.

FIG. 7 shows another embodiment of the invention. FIG. 7 shows a flowdiagram of a method to identify a particular data object in a database.In step 701 a search query is composed to identify a candidate dataobject which is a candidate for the particular data object. In step 702the candidate data object is identified in the database based on asearch query. In step 703 the candidate data object is presented to theuser. In step 704 user feedback on the relevance or irrelevance of asegment of the candidate data object is received. In step 705 animproved candidate data object is identified in the database in responseto the received user feedback. The improved candidate data object is animproved candidate for the particular data object.

The method of FIG. 7 may be implemented in a computer program product.The computer program product comprises computer instructions for causinga processor system to perform the steps of the method of FIG. 7.

It should be noted that the above-mentioned embodiments illustraterather than limit the invention, and that those skilled in the art willbe able to design many alternative embodiments without departing fromthe scope of the appended claims.

In the claims, any reference signs placed between parentheses shall notbe construed as limiting the claim. Use of the verb “comprise” and itsconjugations does not exclude the presence of elements or steps otherthan those stated in a claim. The article “a” or “an” preceding anelement does not exclude the presence of a plurality of such elements.The invention may be implemented by means of hardware comprising severaldistinct elements, and by means of a suitably programmed computer. Inthe device claim enumerating several means, several of these means maybe embodied by one and the same item of hardware. The mere fact thatcertain measures are recited in mutually different dependent claims doesnot indicate that a combination of these measures cannot be used toadvantage.

1. A system (101) for identifying a particular data object of a datatype in a database (104) comprising data objects of the data type, thesystem comprises: a query composition unit (102) for composing a searchquery to identify a candidate data object (206, 207,501) being acandidate for the particular data object, a search unit (103) foridentifying, based on the search query, the candidate data object (206,207, 501) in the database, a presentation unit (105) for presenting thecandidate data object (206, 207, 501) to the user, a feedback receivingunit (106) for receiving user feedback on the relevance or irrelevanceof a segment (301, 302, 303, 304, 401, 402, 504, 505, 506) of thecandidate data object (206, 207, 501), the search unit (103) beingfurther constructed for identifying an improved candidate data object inresponse to the received user feedback, wherein the improved candidatedata object is an improved candidate for the particular data object. 2.A system (101) according to claim 1, wherein the segment (301, 302, 303,304, 401, 402, 504, 505, 506) is formed by a spatial partition or atemporal partition of the candidate data object (206, 207, 501), or thesegment is formed by a combination of the spatial partition and thetemporal partition of the candidate data object.
 3. A system (101)according to claim 2, wherein the spatial partition of the candidatedata object (206, 207, 501) is a two dimensional or three dimensionalspatial partition of the candidate data object (206, 207, 501).
 4. Asystem (101) according to claim 1, wherein the segment (301, 302, 303,304, 403) of the candidate data object (206, 207, 501) is a user definedsegment (301, 302, 303, 304, 403) of the candidate data object (206,207, 501).
 5. A system (101) according to claim 1, which furthercomprises a segment proposal unit (107) for proposing a set of candidatesegments (401, 403, 504, 505, 506) being segments of the candidateobject, wherein the presentation unit is further constructed forpresenting the candidate segments (401, 403, 504, 505, 506) of the setof candidate segments in combination with the candidate data object(207, 501) to the user, and wherein the feedback receiving unit (106) isfurther constructed for receiving an indication from the user which ofthe candidate segments (401, 403, 504, 505, 506) is the segment on whichuser feedback is received by the feedback receiving unit (106).
 6. Asystem (101) according to claim 5, wherein the data objects are medicaldata objects (206, 207, 501), and wherein the segment proposal unit(107) is arranged for proposing the set of candidate segments (401, 402)based on anatomically defined parts of a body.
 7. A system (101)according to claim 5, wherein the segment proposal unit (107) isconstructed for proposing the set of candidate segments (504, 505, 506)based on a partitioning of the candidate data object (206, 207, 501)according to a grid.
 8. A system (101) according to claim 1, wherein thefeedback receiving unit (106) is constructed for receiving user feedbackon the relevance and irrelevance of at least two segments (301, 302,303, 304, 401, 402, 504, 505, 506) of the candidate data object (206,207, 501).
 9. A system (101) according to claim 8, wherein the feedbackreceiving unit (106) is further constructed to receive a ranking of theat least two segments (301, 302, 303, 304, 401, 402, 504, 505, 506) ofthe candidate data object (206, 207, 501), wherein the ranking expresseswhich one of the at least two segments (301, 302, 303, 304, 401, 402,504, 505, 506) is more important for the user than the other one of theat least two segments (301, 302, 303, 304, 401, 402, 504, 505, 506). 10.A system (101) according to claim 1, wherein the feedback receiving unit(106 is constructed for receiving user feedback on the relevance orirrelevant of a feature of the content of the segment (301, 302, 303,304, 401, 402, 504, 505, 506) of the candidate data object (206, 207,501).
 11. A system (101) according to claim 1, wherein theidentification of the improved candidate data object is performed byupdating the search query in response to the received user feedback andby identifying the improved candidate data object based on the updatedsearch query.
 12. A system (101) according to claim 1, wherein theidentification of the improved candidate data object is performed byidentifying a first list of candidate data objects in response to thesearch query, by identifying a second list of candidate data objects inresponse to the received user feedback, and by selecting the improvedcandidate data object from the first list or the second list.
 13. Asystem (101) according to claim 1, wherein the data type is one of thefollowing data types: an image (206, 207, 501), a three dimensionalrepresentation of an object or an environment, a video, or an audiofragment.
 14. A method of identifying a particular data object of a datatype in a database (104) comprising data objects of the data type, themethod comprises the steps of: composing (701) a search query toidentify a candidate data object (206, 207, 501) being a candidate forthe particular data object, identifying (702), based on the searchquery, the candidate data object (206, 207, 501) in the database (104),presenting (703) the candidate data object (206, 207, 501) to the user,receiving (704) user feedback on the relevance or irrelevance of asegment (301, 302, 303, 304, 401, 402, 504, 505, 506) of the candidatedata object (206, 207, 501), identifying (705) an improved candidatedata object in response to the received user feedback, wherein theimproved candidate data object is an improved candidate for theparticular data object.
 15. A computer program product comprisingcomputer instructions for causing a processor system to perform themethod according to claim 14.